Multi-exposure image capturing device and associated system for monitoring a driver

ABSTRACT

The invention relates to an image capturing device (1) comprising an image sensor (9) making it possible to obtain both an infrared image (31) and an image in the visible range (35) by means of an array of elementary optical filters comprising first optical filters, which are at least partially transmissive in the infrared range, and second optical filters which are at least partially transmissive in the visible range. The image capturing device comprises a calculator (12) which is programmed to control the image sensor to perform two shots, one being exposed according to an ambient brightness in the infrared range for obtaining the first image and the other being exposed as a function of an ambient brightness in the visible range for obtaining the second image. The invention also relates to a system for monitoring a driver (3), comprising such an image capturing device.

The present invention relates to an image-capturing device. It alsorelates to a monitoring device for monitoring a driver of a vehicle,including an image-capturing device.

Image-capturing devices that make it possible to view a given scene bothin the visible range and in the infrared range, and to do so with oneand the same image sensor, have been developed recently.

The image sensor of such a device, which is hybrid as it were, issometimes called an “RGB-IR” (acronym of Red Green Blue—InfraRed) imagesensor.

This image sensor comprises a matrix array of photosensitive pixels andan array of elementary optical filters coinciding with these variousphotosensitive pixels.

Some of these elementary optical filters are color filters. They eachtransmit a portion of the visible radiation received by the imagesensor, this portion corresponding in practice to a red, to a green orto a blue. These various elementary color filters make it possible toacquire a color image, for example an “RGB” image.

The other elementary optical filters of the image sensor are at leastpartially transparent in the infrared range. They make it possible toacquire an image of the scene in question in the infrared range.

The color image and the image in the infrared range, which containadditional information, are thus obtained with one and the same sensor,this being beneficial in particular in terms of cost price and size.

However, the ambient brightness in the environment of an image-capturingdevice is often markedly different in the visible range and in theinfrared range. It is therefore generally not possible with such animage sensor to obtain optimum exposure both for the color image and forthe image in the infrared range.

In this context, the present invention proposes an image-capturingdevice comprising an image sensor, which includes:

-   -   an array of optical filters receiving electromagnetic radiation        and comprising first optical filters each able to transmit a        first portion of the electromagnetic radiation contained in a        given infrared wavelength range, and second optical filters each        able to transmit at least one component of a second portion of        the electromagnetic radiation located in the visible, and    -   a matrix array of photosensitive pixels comprising first        photosensitive pixels arranged so as to receive the first        portion of the electromagnetic radiation transmitted by the        first optical filters, as well as second photosensitive pixels        arranged so as to receive the component transmitted by the        second optical filters, each of the first and second        photosensitive pixels producing via the photoelectric effect an        electric charge or an electric voltage depending on the power of        the portion of the electromagnetic radiation that it receives,

According to the invention, the image-capturing device further comprisesa computer programmed to:

-   -   a1) determine an ambient brightness in the infrared range,    -   a2) determine an ambient brightness in the visible range,    -   b1) command the image sensor to perform a first image capture,        during which each of the first photosensitive pixels delivers a        pixel signal representative of the electric charge or of the        electric voltage that it produces, this electric charge or        voltage having been accumulated by this first photosensitive        pixel during a first integration time,    -   b2) command the image sensor to perform a second image capture,        during which each of the second photosensitive pixels delivers a        pixel signal representative of the electric charge or of the        electric voltage that it produces, this electric charge or        voltage having been accumulated by this second photosensitive        pixel during a second integration time,    -   c1) command a preprocessing module to multiply each of the pixel        signals produced in step b1) by a first gain value,    -   c2) command the preprocessing module to multiply each of the        pixel signals produced in step b2) by a second gain value,    -   d1) compose a first image, from the pixel signals produced by        the first photosensitive pixels in the first image capture, said        pixel signals having been multiplied by the first gain value,    -   d2) compose a second image, from the pixel signals delivered by        the second photosensitive pixels in the second image capture,        said pixel signals having been multiplied by the second gain        value,    -   the first integration time or the first gain value being        determined by the computer depending on the ambient brightness        in the infrared range, and    -   the second integration time or the second gain value being        determined by the computer depending on the ambient brightness        in the visible range.

This image-capturing device makes it possible, using the same imagesensor, to obtain:

-   -   in the infrared range, a first image of a scene to be viewed,        which image is suitably exposed since it is exposed depending on        the ambient brightness in the infrared range, and    -   in the visible range, a second image of the same scene, which        image is also suitably exposed since it is exposed depending on        the ambient brightness in the visible range.

This image-capturing device moreover has the advantage of being able tobe implemented by means of commercially available electronic componentsinitially designed for high-dynamic-range (HDR) imagery, thisfacilitating this implementation.

HDR imaging may consist, for example, in:

-   -   acquiring an image with a given exposure time, i.e. with a given        integration time, allowing a suitable exposure of most of this        image, then    -   acquiring a complementary image with another exposure time        allowing a suitable, not overexposed, exposure of the        high-brightness regions of this complementary image, then    -   combining said image and said complementary image to obtain an        HDR image, in which the regions of medium brightness, and those        of high brightness, are suitably exposed, avoiding an over- or        under-saturation of the corresponding brightness values.

Electronic components for HDR imaging are therefore well suited toacquiring (or to commanding an image sensor to acquire) images in seriesof a plurality of images, for example in series of two images, or offour images, the various images of each series being acquired withvarious exposure times or various gains, or both.

It will therefore be understood that such electronic components make itpossible to easily produce the image-capturing device according to theinvention.

Another advantage of this image-capturing device is that it allows suchHDR imaging techniques to be easily applied, when the first image isobtained in the infrared, or the second image is obtained in thevisible.

By way of example, the infrared first image may for example be composedtaking into account both:

-   -   pixel signals produced by the first photosensitive pixels in the        first image capture, and    -   pixel signals produced by the first photosensitive pixels in the        second image capture (this second image capture initially being        intended to obtain the visible second image),    -   this most often allowing the dynamic range of this infrared        first image to be increased, without even having to carry out        additional image captures since the first and second image        captures are generally carried out with different integration        times and/or different gain values.

The following are other non-limiting and advantageous characteristics ofthe image-capturing device according to the invention:

-   -   the computer is programmed, steps b1), c1) and d1) having been        executed beforehand, to determine said ambient brightness in the        infrared range, in step a1), depending on brightness values of        at least some of the image pixels of the first image produced in        said previous execution of step d1);    -   the computer is programmed to, in step a1), determine said        ambient brightness in the infrared range in such a way that it        is representative of a first average-brightness level in the        first image produced in said previous execution of step d1);    -   the computer is programmed to determine the first integration        time or the first gain value depending on a first difference        between, on the one hand, a first target average-brightness        value, and, on the other hand, the first average-brightness        level in the first image produced in the previous execution of        step d1);    -   the computer is programmed, steps b2), c2) and d2) having been        executed beforehand, to determine said ambient brightness in the        visible range, in step a2), depending on brightness values of at        least some of the image pixels of the second image produced in        said previous execution of step d2);    -   the computer is programmed to, in step a2), determine said        ambient brightness in the visible range in such a way that it is        representative of a second average-brightness level in the        second image produced in said previous execution of step d2);    -   the computer is programmed to determine the second integration        time or the second gain value depending on a second difference        between, on the one hand, a second target average-brightness        value, and, on the other hand, the second average-brightness        level in the first image produced in the previous execution of        step d2);    -   the computer is further programmed to:    -   b3) command the image sensor to perform a third image capture,        during which each of the first or second photosensitive pixels        delivers a pixel signal representative of the electric charge or        of the electric voltage that it produces, this electric charge        or voltage having been accumulated by the photosensitive pixel        during a third integration time,    -   c3) command the preprocessing module to multiply each of the        pixel signals produced in step b3) by a third gain value,    -   the product of the third integration time multiplied by the        third gain value being different from the product of the first        integration time multiplied by the first gain value, or being        different from the product of the second integration time        multiplied by the second gain value;    -   the computer is programmed to, in step d1), compose the first        image while further taking into account the pixel signals        produced by the first photosensitive pixels in the third image        capture, these pixel signals having been multiplied by the third        gain value, the product of the third integration time multiplied        by the third gain value being different from the product of the        first integration time multiplied by the first gain value;    -   in step d1), the pixel signals produced by the first        photosensitive pixels in the third image capture, and which have        been multiplied by the third gain value, are further multiplied        by a first scaling coefficient before being taken into account        to compose the first image, the first scaling coefficient        depending on the ratio between the first integration time and        the third integration time, or on the ratio between the first        value gain and the third gain value;    -   the computer is programmed, steps b1), c1), b3), c3) and d1)        having been executed beforehand, to determine the third        integration time or the third gain value depending on brightness        values of at least some of the image pixels of the first image        produced in said previous execution of step d1);    -   the computer is programmed to determine the third integration        time depending on the first integration time;    -   the computer is programmed to determine the third gain value        depending on the first gain value;    -   the computer is programmed to, in step d2), compose the second        image while further taking into account the pixel signals        produced by the second photosensitive pixels in the third image        capture, these pixel signals having been multiplied by the third        gain value, the product of the third integration time multiplied        by the third gain value being different from the product of the        second integration time multiplied by the second gain value;    -   in step d2), the pixel signals produced by the second        photosensitive pixels in the third image capture, and which have        been multiplied by the third gain value, are further multiplied        by a second scaling coefficient before being taken into account        to compose the second image, the second scaling coefficient        depending on the ratio between the second integration time and        the third integration time, or on the ratio between the second        value gain and the third gain value;    -   the computer is programmed, steps b2), c2), b3), c3) and d2)        having been executed beforehand, to determine the third        integration time or the third gain value depending on brightness        values of at least some of the image pixels of the second image        produced in said previous execution of step d2);    -   the computer is programmed to determine the third integration        time depending on the second integration time;    -   the computer is programmed to determine the third gain value        depending on the second gain value;    -   the computer is further programmed to:    -   b4) command the image sensor to perform a fourth image capture,        during which each of the first or second photosensitive pixels        delivers a pixel signal representative of the electric charge or        of the electric voltage that it produces, this electric charge        or voltage having been accumulated by the photosensitive pixel        during a fourth integration time,    -   c4) command the preprocessing module to multiply each of the        pixel signals produced in step b4) by a fourth gain value,    -   the product of the fourth integration time multiplied by the        fourth gain value being different from the product of the second        integration time multiplied by the second gain value,    -   the computer being programmed to, in step d2), compose the        second image while further taking into account the pixel signals        produced by the second photosensitive pixels in the fourth image        capture, these pixel signals having been multiplied by the        fourth gain value;    -   the computer is programmed to, in step d1), compose the first        image while further taking into account pixel signals produced        by the first photosensitive pixels in the second image capture,        the pixel signals produced by the first photosensitive pixels in        the second image capture having been multiplied by the second        gain value;    -   the computer is programmed to, in step d2), compose the second        image while further taking into account pixel signals produced        by the second photosensitive pixels in the first image capture,        the pixel signals produced by the second photosensitive pixels        in the first image capture having been multiplied by the first        gain value.

Of course, the various features, variants and embodiments of theinvention may be combined with one another in various combinations, aslong as they are not mutually incompatible or mutually exclusive.

The invention also proposes a monitoring system for monitoring a driverof a vehicle, comprising an image-capturing device as described aboveand a processing unit programmed to determine a level of the driver'sinability to drive based at least on said first image.

The level of inability to drive may comprise a drowsiness level or alevel of distraction of the driver.

In addition, various other features of the invention will becomeapparent from the appended description, which is given with reference tothe drawings, which illustrate non-limiting embodiments of the inventionand in which:

FIG. 1 schematically shows, seen from the side, a motor vehiclecomprising an image-capturing device implementing the teachings of theinvention,

FIG. 2 shows, in more detail, certain elements of the image-capturingdevice of FIG. 1 ,

FIG. 3 schematically shows, seen face-on, an array of optical filterswith which an image sensor of the image-capturing device of FIG. 1 isprovided,

FIG. 4 schematically shows, seen from the side, the same image sensor,

FIG. 5 schematically shows an infrared image and a color image deliveredby the image-capturing device of FIG. 1 ,

FIG. 6 schematically shows steps of an image-capturing method accordingto a first embodiment implemented by an image-capturing device such asthat of FIG. 1 ,

FIG. 7 schematically shows steps of an image-capturing method accordingto a second embodiment implemented by an image-capturing device such asthat of FIG. 1 , and

FIG. 8 schematically shows steps of an image-capturing method accordingto a third embodiment implemented by an image-capturing device such asthat of FIG. 1 .

I. Image-Capturing Device

FIG. 1 shows a vehicle 5, here a motor vehicle, equipped with amonitoring system 2 for monitoring a driver 3 of the vehicle. Thismonitoring system 2 comprises an image-capturing device 1 and aprocessing unit 20 that is programmed to determine a level of thedriver's 3 inability to drive, I_(L), based on one or more imagesdelivered by the image-capturing device 1.

As shown in FIG. 1 , the image-capturing device 1 is located in apassenger compartment 7 of the vehicle, in a region close to awindshield of the vehicle. It is for example integrated into aninstrument panel, into a dashboard or into a control console of thevehicle.

The image-capturing device 1 is oriented such that its field of view 15covers the region usually occupied by the head of the driver 3 when saiddriver is seated in the driver's seat. The image-capturing device 1 isthus able to capture images of the face of the driver 3.

As shown in FIG. 2 , the image-capturing device 1 comprises:

-   -   an image sensor 9,    -   an optical system 10 such as an objective, which forms on the        image sensor 9 an image of the content of the field of view 15        of the image-capturing device 1,    -   a preprocessing module 11, to amplify pixel signals sp1, sp2        produced by the image sensor 9, then to perform an        analog-digital conversion of these signals in order to obtain a        digital image signal si1, si2,    -   a computer 12, configured to receive the image signal si1, si2        delivered by the preprocessing module 11 and to process this        signal in order to produce one or more images representative of        the content of the field of view 15 of the image-capturing        device 1.

Noteworthily, the image sensor 9 is in some respects a hybrid sensor,allowing the content of the field of view 15 to be viewed both in theinfrared range and in the visible range.

To this end, the image sensor 9 is equipped with a particular array 170of elementary optical filters (FIG. 3 ), comprising an alternation offirst optical filters 171, which are at least partially transmissive inthe infrared range, and of second optical filters 172, 173, 174, whichare at least partially transmissive in the visible range. This array 170of optical filters makes it possible, with the same matrix array 19 ofphotosensitive pixels 21,21′, to obtain both a first image 31, which isan image in the infrared range (called the “infrared image” below), anda second image 35, which is an image in the visible range.

In the example described here, the second optical filters 172, 173, 174are color filters, and the second image in question is a color image.Below, the second image 35 is moreover called the “color image 35”.

By virtue of this particular image sensor 9, the image-capturing device1 is able to acquire both the infrared image 31 and the color image 35,while remaining compact. The infrared image 31 may for example be usedto monitor the driver 3 and to determine his level of inability to driveI_(L). The color image 35 is more pleasing for the human eye to view andcontains more information than an infrared image (usually displayed ingrayscale). The color image may therefore be acquired for multiplepurposes; for example, it may be used to communicate with a remoteelectronic device, for example within the context of a teleconference,or indeed be kept in memory for security reasons, or as a “souvenirphoto” for commemorating a journey.

When the image-capturing device 1 takes a shot:

-   -   each photosensitive pixel 21, 21′ of the matrix array 19 of the        image sensor accumulates, during a given integration time, an        electric charge or voltage generated via the photoelectric        effect in this photosensitive pixel, and delivers a pixel signal        representative of the electric charge or voltage thus        accumulated, then    -   the preprocessing module 11 multiplies each of these pixel        signals by a given gain value, then carries out analog-digital        conversion of the pixel signals thus amplified (or possibly        attenuated).

Whether such a shot results in an image (or more than one images) thatis (are) suitably exposed depends on the exposure parameters with whichthis shot is taken. These exposure parameters comprise the integrationtime and gain value mentioned above, and the aperture of a diaphragm ofthe optical system 10 (not shown in the figures). The aperture of thisdiaphragm is here considered to remain constant in the series of shotsdescribed below (each series of shots being taken in a short time).Thus, the only exposure parameters considered below are the integrationtime and gain value in question.

Noteworthily, the computer 12 is programmed to command the image sensor9 and the preprocessing module 11 in such a way as to take shots inseries of a plurality of shots—for example, in pairs, or in series of 3or 4 shots. In other words, the computer 12 is programmed so that eachshot taken by the image-capturing device is necessarily accompanied byat least one other shot, associated with the previous one. These twoshots are here taken in a short time, for example shorter than 0.3seconds.

Each series of shots commanded by the computer 12 comprises at least:

-   -   a first shot, taken with a first integration time ti1 (i.e. with        a first exposure time), and with a first gain value g1, the        first integration time ti1 or the first gain value g1 being        determined by the computer 12 depending on an ambient brightness        in the infrared range, and    -   a second shot, taken with a second integration time ti2 (with a        second exposure time), and with a second gain value g2, the        second integration time ti2 or the second gain value g2 being        determined by the computer 12 depending on an ambient brightness        in the visible range.

The computer 12 is programmed to then compose the infrared image 31 onthe basis of the pixel signals sp1 acquired during the first shot, andto compose the color image 35 on the basis of the pixel signals sp2acquired during the second shot.

By virtue of this multi-shot technique, the infrared image 31 and thecolor image 35 are each suitably exposed, even if the ambient brightnessin the field of view 15 of the image-capturing device 1 is markedlydifferent in the visible range and in the infrared range. Twocomplementary images of the content of this field of view 15,corresponding to different wavelength bands, and each suitably exposed,are thus obtained.

The computer 12 is more precisely programmed to:

-   -   a1) determine the ambient brightness in the infrared range,    -   a2) determine the ambient brightness in the visible range,    -   b1) command the image sensor 9 to perform a first image capture,        during which each of the first photosensitive pixels 21 delivers        a pixel signal representative of the electric charge or electric        voltage that it produces, this electric charge or voltage having        been accumulated by this first photosensitive pixel 21 during        the first integration time ti1 (in the case where the image        sensor is of CCD type, the pixel signal in question corresponds        for example directly to the electric charge accumulated via the        photoelectric effect by the photosensitive pixel in question, at        the end of the integration time in question),    -   b2) command the image sensor 9 to perform a second image        capture, during which each of the second photosensitive pixels        21′ delivers a pixel signal representative of the electric        charge or of the electric voltage that it produces, this        electric charge or voltage having been accumulated by this        second photosensitive pixel 21′ during the second integration        time ti2,    -   c1) command the preprocessing module 11 to multiply each of the        pixel signals sp1 produced in step b1) by a first gain value g1,    -   c2) command the preprocessing module to multiply each of the        pixel signals sp2 produced in step b2) by a second gain value        g2,    -   d1) compose the infrared mage 31, from the pixel signals sp1        produced by the first photosensitive pixels 21 in the first        image capture, said pixel signals sp1 having been multiplied by        the first gain value g1,    -   d2) compose the second image 35, which it will be recalled is        here a color image, from the pixel signals sp2 delivered by the        second photosensitive pixels 21′ in the second image capture,        said pixel signals sp2 having been multiplied by the second gain        value g2.

Various embodiments of the image-capturing device 1 are described here.These embodiments mainly differ from one another in the detail of theprogramming of the computer 12, i.e. in the detail of theimage-capturing method implemented by the image-capturing device (stepsa1) to d2) of which have just been presented). In the present case, themethods schematically shown in FIGS. 6, 7 and 8 correspond to a first,second and third embodiment of the image-capturing device 1,respectively.

In the first and third embodiments (FIGS. 6 and 8 ), the computer 12 isprogrammed so that each series of shots comprises two shots (in thepresent case the first and second shots presented above), whereas in thesecond embodiment (FIG. 7 ), the computer 12 is programmed so that eachseries of shots comprises four shots (taken with different exposureparameters).

Realization of this image-capturing device 1 is facilitated by the factthat electronic components, initially designed for HDR imaging, andtherefore basically configured to take multi-exposure shots, are nowcommercially available. Moreover, electronic components for HDR imagingare generally configured to take shots in series of two, or four shots,and are therefore particularly well suited to the implementation of thethree embodiments mentioned above.

In the second and third embodiments (unlike the first embodiment), eachof the first and second images 31, 35 is moreover obtained by furtherimplementing an HDR imaging technique.

From one embodiment to the next, identical or corresponding elementshave as much as possible been identified by the same reference signs andare not described each time.

The image sensor 9 is now described in more detail. The way in which theinfrared image and the color image are composed, and the way in whichthe exposure parameters are controlled, will be presented subsequently,in the description of various embodiments of an image-capturing methodimplemented by the image-capturing device 1.

RGB-IR Image Sensor.

As may be seen in FIGS. 3 and 4 , each optical filter 171, 172, 173, 174of the array 170 of optical filters of the image sensor 9 is placedfacing one of the photosensitive pixels 21, 21′ of the matrix array 19,so as to filter electromagnetic radiation coming from the field of view15 of the image-capturing device 1 (radiation that was collected by theoptical system 10), before this radiation reaches the photosensitivepixels 21, 21′ in question. Each photosensitive pixel 21, 21′ thusreceives one portion of the electromagnetic radiation, which portion hasbeen filtered by the optical filter 171, 172, 173, 174 with which it isassociated (this optical filter forms a filtering element, as regardsthe electromagnetic radiation that it receives).

The first optical filters 171 are each able to transmit a first portionof the electromagnetic radiation, which portion is located in a givenwavelength range of the infrared. In the embodiments described here, thefirst optical filters 171 mainly transmit wavelengths located in thiswavelength range of the infrared. This wavelength range mainly liesbeyond 700 nanometers. It may for example extend from 700 nanometers to1100 nanometers.

As regards the second optical filters 172, 173, 174, they are each ableto transmit at least one component of a second portion of theelectromagnetic radiation, which portion is located in the visible,between 400 nanometers and 700 nanometers.

In the example described here, the second optical filters 172, 173, 174,mainly transmit wavelengths located between 400 and 700 nanometers. As avariant, however, they could transmit both wavelengths located in thevisible and wavelengths located in the infrared.

The second optical filters 172, 173, 174 here comprise red opticalfilters 172, green optical filters 173 and blue optical filters 174. Theterms “red”, “green” and “blue” are used with their common meaning. Thevalues of the red, green and blue passbands given below are given by wayof non-limiting example.

The red optical filters 172 here have a red passband comprised mainlybetween 550 nm and 700 nm. The component of the second portion of theelectromagnetic radiation which they transmit therefore mainly consistsof wavelengths comprised between 550 nm and 700 nm.

As for the green optical filters 173, they here have a green passbandmainly comprised between 450 nm and 650 nm, while the blue opticalfilters 174 have a blue passband mainly comprised between 400 nm and 550nm.

The various optical filters 171, 172, 173, 174 of the array of opticalfilters are arranged with respect to one another so as to form a pattern175 that is repeated regularly so as to form the array 170 of opticalfilters. This pattern 175 here comprises four optical filters, in thepresent case: one of the first optical filters 171, one of the redoptical filters 172, one of the green optical filters 173, and one ofthe blue optical filters 174. These four adjacent optical filters form asquare. As shown in the figures, the array 170 of optical filters istherefore comparable to a so-called “Bayer” array one of the greenoptical filters of which has been replaced with an element that istransparent in the infrared. As a variant, the pattern of opticalfilters, which pattern is repeated a plurality of times to form thearray of optical filters, could be formed differently (for example so asto comprise more green optical filters than red or blue opticalfilters).

As regards the photosensitive pixels 21, 21′ of the matrix array 19,they comprise:

-   -   the first photosensitive pixels 21, each arranged so as to        capture the first portion of the electromagnetic radiation        transmitted by one of the first optical filters 171, and    -   the second photosensitive pixels 21′, each arranged so as to        capture the red, green or blue component transmitted by one of        the second optical filters 172, 173, 174.

Each of the first and second photosensitive pixels 21, 21′ produces, viathe photoelectric effect, an electric charge or an electric voltage thatdepends on the power of the portion of the electromagnetic radiationthat it receives.

In the embodiment described here, the semiconductor substrate of thematrix array 19 of photosensitive pixels 21, 21′ is made of (suitablydoped) silicon. The sensitivity of the photosensitive pixels in theinfrared range is thus limited to the near-infrared range: thesensitivity range of the first photosensitive pixels 21, equipped withthe first optical filters 171, is here comprised between 700 nanometersand 1100 nanometers.

The image sensor 9 is for example a CMOS image sensor (CMOS being theacronym of complementary metal-oxide semiconductor) or CCD image sensor(CCD being the acronym of charge-coupled device).

With respect now to the computer 12, it comprises at least a processorand an electronic memory. The computer 12 here comprises:

-   -   a processing module 13, configured to receive the various image        signals si1, si2 delivered by the preprocessing module 11, and        to process these signals in order to produce the infrared image        31 and the color image 35, and    -   a control module 14, for controlling the various exposure        parameters of the image-capturing device 1 and for commanding        the image sensor 9 and the preprocessing module 11 to take the        one or more series of shots mentioned above.

II. Image-Capturing Method

The computer 12 may be programmed so that the image-capturing device 1implements an image-capturing method according to a first, a second or athird embodiment, corresponding to FIG. 6 , to FIG. 7 and to FIG. 8 ,respectively. Each of these figures shows the main steps executed by thecomputer 12 during the image-capturing method.

As explained above, whatever the embodiment in question, the method inparticular comprises:

-   -   a series of shots comprising at least the first and second shots        mentioned above, these shots being tailored to the ambient        brightness in the infrared and to the ambient brightness in the        visible, respectively, and    -   composition of the infrared image 31 and of the color image 35,        on the basis of the image signals si1, si2 obtained in these        first and second shots.

The way in which this series of shots is taken will first be described.Composition of the infrared image 31 and of the color image 35 is thendescribed, before the way in which the exposure parameters arecontrolled in these shots is presented.

Series of Shots

In the first and third embodiment (FIGS. 6 and 8 ), the computer 12commands the image sensor 9 and the preprocessing module 11 to takeshots in series of two shots, each series comprising the first andsecond shots mentioned above.

The computer 12 commands the first shot in steps b1) and c1) mentionedabove. This first shot comprises, in addition to the acquisition of thepixel signals sp1 produced by the photosensitive pixels 21,21′,multiplication of these pixel signals sp1 by the first gain value g1then their conversion to a digital format to obtain a digital firstimage signal si1 (which collates all the pixel signals, amplified orattenuated, then converted into digital format).

In the same way, the computer 12 commands the second shot in steps b2)and c2). This second shot makes it possible to obtain a second imagesignal si2, which collates all the pixel signals sp2 produced in thesecond image capture, after multiplication by the second gain value g2and conversion to digital format.

In the second embodiment (FIG. 7 ), the computer 12 commands the imagesensor 9 and the preprocessing module 11 to take shots in series of fourshots, each series comprising the first and second shots describedabove, as well as a third and fourth shots.

The computer 12 commands the third shot in the following steps, stepsb3) and c3):

b3) commanding the image sensor 9 to perform a third image capture,during which each of the first and second photosensitive pixels 21, 21′delivers a pixel signal sp3 representative of the electric charge or ofthe electric voltage that it produces, this electric charge or voltagehaving been accumulated by the photosensitive pixel 21, 21′ during athird integration time ti3, and

-   -   c3) commanding the preprocessing module to multiply each of the        pixel signals sp3 produced in step b3) by a third gain value g3,        then to convert these (amplified or attenuated) pixel signals to        digital format to obtain a digital third image signal si3.    -   The third shot is taken with a set of exposure parameters that        is different and complementary to that used for the first shot.        Specifically, this third shot is taken with the aim of then        combining the third image signal si3 with the first image signal        si1 in order to increase the dynamic range of brightness values        of the infrared image 31, and thus of ensuring not only that the        darkest regions of this image are not underexposed, but also        that the brightest regions of this image are not overexposed.

In the present case, the third exposure time ti3 and the third gainvalue g3 are such that the product ti3·g3 (product of the thirdintegration time ti3 multiplied by the third gain value g3), whichcorresponds to an effective exposure time of the third shot, isdifferent from the product ti1·g1 (product of the first integration timeti1 multiplied by the first gain value g1).

Comparably, the computer 12 commands the fourth shot in the followingsteps, steps b4) and c4):

-   -   b4) commanding the image sensor 9 to perform a fourth image        capture, during which each of the first and second        photosensitive pixels 21, 21′ delivers a pixel signal sp4        representative of the electric charge or of the electric voltage        that it produces, this electric charge or voltage having been        accumulated by the photosensitive pixel 21, 21′ during a fourth        integration time ti4, and    -   c4) commanding the preprocessing module to multiply each of the        pixel signals sp4 produced in step b4) by a fourth gain value        g4, then to convert these (amplified or attenuated) pixel        signals to digital format to obtain a digital fourth image        signal si4.

The fourth shot is taken with a set of exposure parameters that isdifferent and complementary to that used for the second shot.Specifically, this fourth shot is taken with the aim of combining thefourth image signal si4 with the second image signal si2 to increase thedynamic range of brightness values of the color image 35, and ofensuring not only that the darkest regions of this image are notunderexposed, but also that the brightest regions of this image are notoverexposed. More precisely, the fourth exposure time ti4 and the fourthgain value g4 are such that the product ti4·g4 (product of the fourthintegration time ti4 multiplied by the fourth gain value g4) isdifferent from the product ti2·g2 (product of the second integrationtime ti2 multiplied by the second gain value g2).

Within each image signal si1, si2, si3, si4, a (numerical) brightnessvalue is associated with each photosensitive pixel 21, 21′ of the imagesensor 9, this value being representative of the power of the portion ofthe electromagnetic radiation captured by this photosensitive pixel,i.e. the portion accumulated during the exposure time ti1, ti2, ti3, ti4employed in the corresponding image capture.

As regards the three embodiments shown in the figures, in each imagecapture (whether it is a question of the first, second, third orfourth), both the first photosensitive pixels 21 and the secondphotosensitive pixels 21′ accumulate the electric charges or voltagesthat they produce, during the same integration time ti1, ti2, ti3, ti4(during the integration time corresponding to the image capture inquestion), then deliver the pixel signals mentioned above.

In other embodiments provision could however be made for the first imagecapture and the second image capture to be carried out via a singleexposure of all of the image sensor 9, during which exposure the firstphotosensitive pixels 21 would accumulate the electric charges orvoltages that they produce during a time equal to the first integrationtime ti1, while, in parallel, the second photosensitive pixels 21′ wouldaccumulate the electric charges or voltages that they produce during atime equal to the second integration time ti2 (i.e. using a technique ofdual exposure of a single image capture comparable to single-shot HDRtechniques).

In any case, when the various image captures in the series of shots aretaken at different successive times, these times are preferably close toone another, for example being separated pairwise by less than 0.3seconds, or even less than 0.1 seconds. This allows the obtainedinfrared image 31 and color image 35 to both correspond to the samesituation and attitude of the driver, even in the event of movement ofthe latter.

Composition of the Infrared Image and of the Color Image

In the three embodiments described here, the infrared image 31 is formedfrom a matrix array of image pixels 33 (FIG. 5 ) associated with thevarious photosensitive pixels 21, 21′ of the image sensor 9. Thecomputer 12 is programmed to implement an interpolation algorithm thatmakes it possible to form a “complete” infrared image 31, even thoughonly one photosensitive pixel 21 in four captures infrared radiation, inthe image sensor 9. In other words, the infrared image 31 here comprisesas many image pixels 33 as the image sensor 9 comprises photosensitivepixels, first and second photosensitive pixels 21, 21′ taken together.

In step d1), the computer 12 composes the infrared image 31, inparticular from the brightness values that, in the first image signalsi1, are associated with the first photosensitive pixels 21. For eachimage pixel 33 of the infrared image 31, the computer determines an(interpolated) brightness value, depending on the brightness values ofthe image signal si1 that are associated with the first photosensitivepixels 21 that, in the image sensor 9, are closest to a positioncorresponding, in this sensor, to the image pixel 33 in question.

The color image 35 is also formed from a matrix array of image pixels(not referenced in FIG. 5 ) associated with the various photosensitivepixels 21, 21′ of the image sensor 9. Just as for the infrared image,the computer 12 is here programmed to implement an interpolationalgorithm that makes it possible to form a “complete” color image 35,even though only one photosensitive pixel 21 in four captures radiationlocated in the aforementioned red or green or blue passband. In otherwords, the color image here comprises as many image pixels as the imagesensor 9 comprises photosensitive pixels, first and secondphotosensitive pixels 21, 21′ included.

The color image 35 may, as here, comprise three monochromic channels,i.e. a red channel 37, a green channel 39 and a blue channel 41,respectively. Each of these channels is a grayscale image (each imagepixel of the channel in question has a brightness value, but no hue orchrominance value) associated with the color of the channel in questionand of same size as the color image 35 (i.e. comprises the same numberof image pixels).

In step d2), the computer 12 composes:

-   -   the red channel 37 of the color image 35 from the brightness        values that, in the second image signal si2, are associated with        the second photosensitive pixels 21′ located facing the red        optical filters 172,    -   the green channel 39 of this image from the brightness values of        the second image signal si2 that are associated with the second        photosensitive pixels 21′ located facing the green optical        filters 173, and    -   the blue channel 41 of this image from the brightness values of        the second image signal si2 that are associated with the second        photosensitive pixels 21′ located facing the blue optical        filters 174.

To generate the color image 35, the computer 12 may for example make acombination the brightness values of the red channel 37, of the greenchannel 39 and of the blue channel 41, for each image pixel, in order toobtain, for the “color” image pixel in question:

-   -   an overall brightness value for the visible range (for example        equal to the average of the luminosities of the various        channels), and    -   two complementary chrominance values representative of the        chromatic properties of the captured light, at the point in        question of the image sensor 9.

In other embodiments, the infrared image and/or the color image couldcorrespond to raw images, i.e. images obtained without interpolation. Inthis case, the number of image pixels of the infrared image, forexample, would be equal to the number of said first photosensitivepixels rather than being equal to the total number of photosensitivepixels of the image sensor.

In the first embodiment (FIG. 6 ), in step d1), the computer 12 composesthe infrared image 31 on the basis of the first image signal si1,without taking into account the second image signal si2 or any otherimage signal. Furthermore, in step d2), the computer 12 composes thecolor image 35 only on the basis of the image signal si2, without takinginto account the first image signal si1 or any other image signal.

In the second and third embodiments, the computer also implements a HDRimaging technique.

In the present case, in the second embodiment (FIG. 7 ), in step d1),the computer 12 composes the infrared image 31 taking into account both:

-   -   brightness values that, in the first image signal si1, are        associated with the first photosensitive pixels 21, and    -   brightness values that, in the third image signal si3, are        associated with the first photosensitive pixels 21.

The brightness values of the first image signal si1 (which correspond tothe first integration time ti1) are combined with those of the thirdimage signal si3 (corresponding to the third integration time ti3) sothat, in the end, the dynamic range of the brightness values of theinfrared image 31 is higher than would be possible with a single shot.

By way of example, the third shot is considered here to correspond to aneffective exposure time shorter than that used for the first shot,allowing a suitable exposure of the high-brightness regions of the imageto be obtained (whereas the first shot corresponds to an intermediateeffective exposure, allowing a suitable exposure of the intermediate-and low-brightness regions of the image to be obtained). In other words,the effective exposure time ti3·g3 is here considered to be shorter thanthe effective exposure time ti1·g1 (and, therefore, in the case wherethe gain values g1 and g3 are equal, the third integration time ti3 isshorter than the first integration time ti1).

Before the brightness values of the third image signal si3 are combinedwith those of the first image signal si1, these brightness values arescaled, i.e. made comparable, in order to take into account thedifference in exposure time between the first and third shots. To dothis, each brightness value of the third image signal si3 may forexample be multiplied by a first scaling coefficient equal to(ti1·gi1)/(ti3·gi3) (i.e. equal to the product of the first integrationtime ti1 multiplied by the first gain value g1, divided by the productof the third integration time ti3 multiplied by the third gain valueg3), in order to obtain a scaled third image signal, si3′.

The computer 12 may then, for example:

-   -   compose a first version of the infrared image, 31′, on the basis        of the first image signal si1, as explained above (at the start        of the section relative to composition of the infrared image and        of the color image),    -   in the same way, compose a second version of the infrared image,        31″, on the basis of the scaled third image signal si3′, and    -   fuse the first and second versions of the infrared image 31′,        31″ to obtain the final version of the infrared image, 31.

This fusion may for example be carried out by computing, image pixel byimage pixel, an average between the first version 31′ and the secondversion 31″ of the infrared image.

This fusion may also be carried out by assigning to each image pixel 33of the infrared image 31 a brightness value:

-   -   equal to the brightness value of the corresponding image pixel        33′ (i.e. of the image pixel located at the given position) of        the first version of the infrared image 31′, when the latter        brightness value is below a given brightness threshold, and    -   otherwise equal to the brightness value of the corresponding        image pixel 33″ of the second version of the infrared image 31″.

This brightness threshold may for example be slightly lower than thebrightness value corresponding to an over-saturation (overexposed pixel)in the first version of the infrared image 31′. It may for example becomprised between half of this over-saturation value, and thisover-saturation value (this over-saturation value being equal forexample to 255 when the brightness value of an image pixel of the firstversion of the infrared image is represented by an integer number codedon 8 bits).

The fusion of the first and second versions of the infrared imageprevents, as regards the infrared image 31 obtained in the end, not onlythe dark regions and the regions of intermediate brightness of thisimage from being underexposed (i.e. these regions are notunder-saturated, i.e. of a brightness value equal to 0), but also thebrightest regions of this image from being overexposed (i.e.over-saturated).

Different ways of combining the first and third image signals si1 andsi3 to obtain the infrared image 31 may be envisioned, in particular ifthe third shot corresponds to an effective exposure time ti3·g3 that islonger than the effective exposure time ti1·g1 of the first shot(instead of being shorter than the latter).

In this second embodiment, in step d2), the computer 12 composes thecolor image 35 taking into account both:

-   -   brightness values that, in the second image signal si2, are        associated with the second photosensitive pixels 21′, and    -   brightness values that, in the fourth image signal si4, are        associated with the second photosensitive pixels 21′.

The color image 35 is obtained by combining the second and fourth imagesignals si2, si4 using an HDR imaging technique comparable to the onepresented above with respect to step d1).

By way of example the red channel 37 of the color image 35 may beobtained in an identical manner to the way presented above with respectto the infrared image 31, but based on the brightness values that, inthe image signals si2 and si4, correspond to the red optical filters172, instead of based on the brightness values that, in the imagesignals si1 and si3, correspond to the first optical filters 171, whichare transmissive in the infrared. Furthermore, likewise, the greenchannel 39 and blue channel 41 of the color image 35 may be obtained inthe same way as the infrared image 31, but based on the brightnessvalues that, in the image signals si2 and si4, correspond to the greenand blue optical filters 173, 174, respectively, instead of based on thebrightness values that, in the image signals si1 and si3, correspond tothe first optical filters 171.

In the third embodiment (FIG. 8 ), the infrared image 31 and the colorimage 35 are obtained by implementing an HDR imaging technique, directlyon the basis of the first and second image signals si1 and si2, herewithout taking into account any other image signals.

This third embodiment takes advantage of the fact that the first andsecond shots are generally taken with different effective exposure timesti1·gi1 and ti2·gi2, since the ambient brightness in the visible rangeis generally different from that in the infrared range.

After these two shots, the computer 12 therefore has available to it thetwo image signals si1 and si2, which here both comprise brightnessvalues acquired by the “infrared” first photosensitive pixels 21, andwhich were obtained with two generally different effective exposuretimes. The two image signals si1 and si2 therefore together allow theinfrared image 31 to be composed in such a way that it has a highdynamic range as regards brightness values, without it being necessaryto take additional shots. Furthermore, these two image signals likewiseallow the color image 35 to be composed in such a way that it has a highdynamic range as regards brightness values, without it being necessaryto take additional shots.

In this third embodiment, the infrared image 31 is for example composedas was described above with respect to the second embodiment, but basedon the first and second image signals si1 and si2, instead of based onthe first and third image signals si1 and si3. Thus, before beingcombined with the first image signal, the second image signal si2 is forexample multiplied by the quantity (ti1·g1)/(ti2·g2), so as to make itsbrightness values comparable to those of the first image signal si1.

Furthermore, the color image 35 is for example composed as was describedabove with respect to the second embodiment, but based on the second andfirst image signals si2 and si1, instead of based on the second andfourth image signals si2 and si4.

Control of the Exposure Parameters

As already indicated, whatever the embodiment in question, the computer12 determines the first exposure time ti1 and/or the first gain value g1employed in steps b1) and c1) depending on the ambient brightness in theinfrared range determined in step a1). The computer 12 also determinesthe second exposure time ti2 and/or the second gain value g2 dependingon the ambient brightness in the visible range determined in step a2).

The way in which the ambient brightness in the infrared range and in thevisible range is determined is presented below first, and the way inwhich the exposure parameters are determined is then described.

Determination of the Ambient Brightness in the Infrared Range and in theVisible Range: Steps a1) and a2).

The ambient brightness in the infrared range is representative of thepower of infrared electromagnetic radiation coming from an environmentof the image-capturing device 1, in the present case coming from thefield of view 15 of this device, and received by a brightness sensor. Inthe three embodiments described here, this brightness sensor is theimage sensor 9.

Likewise, the ambient brightness in the visible range is representativeof the power of visible electromagnetic radiation coming from anenvironment of the image-capturing device 1, in the present case comingfrom the field of view 15 of this device, and received by a brightnesssensor, which once again takes the form of the image sensor 9.

In the three embodiments described here, the ambient brightness in theinfrared range is determined, in step a1), from the infrared image 31produced in a previous execution of step d1), and thereforecorresponding to a first shot taken previously by the image sensor 9.

In the present case, the computer 12 determines the ambient brightnessin the infrared range depending on the brightness values of at leastsome of the image pixels 33 of this infrared image 31 previouslyacquired by the image-capturing device 1, for example by computing anaverage of the brightness values of these image pixels 33. The ambientbrightness in the infrared range is then representative of anaverage-brightness level in this infrared image 31, which level iscalled the first brightness level and denoted L_(IR) below.

The average in question may be of all the image pixels 33 of theinfrared image 31, or be of only certain of these image pixels, theseimage pixels being located in a region of interest of the image, forexample a region corresponding to the image of the face of the driver 3.

Moreover, provision may be made for this average to take into accountonly those brightness values of the image pixels that meet a givencriterion, for example those that are below a brightness thresholdvalue. This threshold value may for example be equal to the brightnessthreshold mentioned above (threshold employed in the combination of thefirst and third image signals si1 and si3). The first brightness levelLu is then representative of the average brightness of the regions oflow brightness and of intermediate brightness of the infrared image 31.The latter case is particularly well suited to the second embodiment,since, in the case considered above (ti3·g3<ti1·g1), the first shot aimsto obtain an exposure suitable for dark regions and regions ofintermediate brightness.

As regards the ambient brightness in the visible range, it isdetermined, in step a2), from the color image 35 produced in a previousexecution of step d2), and therefore corresponding to a second shottaken previously by the image sensor 9.

The computer 12 determines this ambient brightness in the visible rangedepending on the brightness values of at least some of the image pixelsof this color image 31 previously acquired by the image-capturing device1, for example by computing an average of the brightness values of theimage pixels of this image. The ambient brightness in the infrared rangeis then representative of an average brightness level in the previouslyacquired color image 35, which level is called the second brightnesslevel and denoted L_(V) below. Just as with the first brightness levelLm, provision may be made for the average in question to be of all theimage pixels of the color image or of certain of these image pixels onlyand/or for this average to take into account only those brightnessvalues of the image pixels that meet a given criterion.

The control of the exposure parameters as such, which is carried out onthe basis of these first and second brightness levels L_(IR) and L_(V),will now be presented.

For the sake of the simplicity of the description, the calculator 12 ishere considered to determine, or in other words adjust, the firstexposure time ti1 and the second exposure time ti2 depending on theambient brightness L_(IR) in the infrared range and on the ambientbrightness L_(V) in the visible range, respectively, whereas the firstand second gain values g1 and g2 are set, and for example equal to eachother. It will however be understood that the computer could,comparably, be programmed to determine (to adjust) the first and secondgain values g1 and g2 depending on the aforementioned brightness levels,while keeping the first and second integration times ti1 and ti2constant (for example at values low enough that the obtained imagesremain sharp even in the event of movement of the head of the driver 3).The computer could also be programmed to determine both these gainvalues and these exposure times depending on the aforementionedbrightness levels, which are dependent on the aforementioned ambientbrightnesses.

In the three embodiments described here, the computer 12 determines thefirst integration time ti1, employed in the first shot, depending on afirst difference el between, on the one hand:

-   -   a first target value L_(IR, O) of medium brightness, and, on the        other hand,    -   the average first brightness level L_(IR) in the infrared image        31 produced in the previous execution of step d1).

More precisely, the computer corrects, or in other words updates, thefirst integration time ti1 on the basis of the first difference ε1. Thiscorrection is made so as to gradually bring the first brightness levelLa to the first target value L_(IR, O), during repetitions of all ofsteps a1), b1), c1) and d1). This correction may consist for example inadding a corrective term to a previous value of the first integrationtime ti1, this corrective term being proportional to the firstdifference ε1 (proportional correction). More generally, this correctionconsists in automatically controlling the first brightness level L_(IR)to the first target value L_(V, O). This automatic control may inparticular be proportional, proportional-integral orproportional-integral-derivative (PID). Such an automatic control of theexposure conditions is sometimes called automatic exposure control (AEC)in the specialist literature.

The first target value L_(IR, O) may correspond to the averagebrightness in an image considered to be suitably exposed, the firsttarget value then for example being comprised in a given intervalextending from one quarter to three quarters of the maximum brightnessvalue that may be associated with an image pixel.

As regards the second integration time ti2 employed in the second shot,it is here determined depending on a second difference ε2 between, onthe one hand:

-   -   a second target value L_(V, O) of medium brightness (for example        equal to the first target value L_(IR, O)), and, on the other        hand,    -   the average second brightness level L_(V) in the color image 35        produced in the previous execution of step d2).

The way in which the second integration time ti2 is determined dependingon the second difference ε2, and the way in which the first integrationtime ti1 is determined depending on the first difference ε1 (in order tobring these differences to zero) are here identical.

In the case of the second embodiment (FIG. 7 ), the computer 12 alsodetermines the third and fourth integration times ti3 and ti4 and thethird and fourth gain values g3 and g4 employed in the third and fourthshots.

As above, for the sake of the simplicity of the description, thecomputer 12 is here considered to adjust the third and fourth exposuretimes ti3 and ti4 depending on the brightness conditions in theenvironment of the image-capturing device 1 or depending on other shotconditions, whereas the third and fourth gain values g3 and g4 are set(for example equal to each other, and possibly equal to the gain valuesg1 and g2). It will however be understood that the computer could beprogrammed to also determine (to adjust) the third and fourth gainvalues g3 and g4 depending on these brightness or shot conditions, or todetermine both the exposure times ti3 and ti4 and the gain values g3 andg4 depending on the conditions in question.

The third integration time ti3 may be determined by the computer 12depending on the first integration time ti1, for example so as to beequal to the product of the first integration time ti1 multiplied by aconstant coefficient. If the third shot is intended to obtain anon-overexposed exposure of high-brightness regions, the coefficientwill be chosen to be lower than 1, and for example equal to one tenth orone quarter (the third integration time ti3 then being equal to afraction of the first integration time ti1). The fourth integration timeti4 may be determined depending on the second integration time ti2, inthe same way, by multiplying the second integration time ti2 by aconstant coefficient.

As a variant, provision may also be made for the third integration timeti3 to be determined, via automatic control (automatic exposurecontrol), as explained above, so as to bring a specific brightnesslevel, of the second version of the infrared image 31″ or of the finalversion of the infrared image 31, to a given specific brightness value.This specific brightness level is for example equal to an average of thebrightness values of the image pixels 33 of the infrared image 31obtained in a previous execution of the step d1). Here, this averagetakes into account:

-   -   only brightness values below a low brightness threshold, when        the third shot is intended to obtain a suitable,        non-underexposed, exposure of the low-brightness regions of the        infrared image (the specific brightness level mentioned above        then being representative of the average brightness of the        low-brightness regions of this image), or    -   only the brightness values above a high brightness threshold,        when the third shot is intended to obtain a suitable,        non-overexposed, exposure of the high-brightness regions of the        infrared image (the specific brightness level then being        representative of the average brightness of the high-brightness        regions of this image).

The fourth integration time ti4 may be determined in a comparable way tothe third integration time ti3, via automatic control, so as to bring aspecific brightness level of the color image 35 to a specific brightnessvalue (this specific brightness level for example being representativeof the average brightness of the high-brightness regions of the colorimage 35).

Lastly, as may be seen in FIGS. 6 to 8 , in these three embodiments, theset of steps comprising steps b1), b2), c1), c2), d1), d2) and possiblyb3), c3), b4) and c4) (in the second embodiment), is executed aplurality of times in succession. Step a1) is for its part also executeda plurality of times in succession (each time on the basis of the resultof a previous execution of step d1)), so as to continuously update thefirst exposure time and/or the first gain value depending on the ambientbrightness in the infrared. Step a2) is for its part also executed aplurality of times in succession (each time on the basis of the resultof a previous execution of step d2)), so as to continuously update thesecond exposure time and/or the second gain value depending on theambient brightness in the visible.

III. Monitoring System for Monitoring the Driver

The electronic processing unit 20 of the monitoring system 2 isprogrammed to determine the level of the driver's 3 inability to driveI_(L) based on at least one of the infrared images 31 produced by theimage-capturing device 1. The level of inability to drive I_(L)comprises for example a drowsiness level of the driver 3 and/or adistraction level of the driver 3 (the level of inability to drive I_(L)may in particular be a drowsiness level of the driver 3 or a distractionlevel of said driver).

The processing unit 20 may for example be programmed so as to analyzethe infrared image 31 in question, or a sequence of infrared images 31produced by the image-capturing device 1, in order to identify the faceof the driver 3 and/or certain regions of the face of the driver 3, inparticular the regions of the infrared image 31 corresponding to theeyes of the driver 3. The processing unit 20 may then determine thedrowsiness level of the driver 3 by measuring the duration and/or thefrequency of the blinks of the eyes of the driver 3, which will havebeen identified beforehand in the infrared image 31.

The processing unit 20 may determine the distraction level of the driver3 on the basis of a posture of the head of the driver 3, which isdeduced from the infrared image 31, and on the basis of the variation inthis posture overtime.

The processing unit 20 may also evaluate (via analysis of the infraredimage 31, or of a sequence of infrared images 31) the gaze direction ofthe driver 3 or the variation in this gaze direction over time, and usethis gaze direction or its variation to determine the distraction leveland/or the drowsiness level of the driver.

The processing unit 20 may also evaluate (via analysis of the infraredimage 31, or of a sequence of infrared images 31) the diameter of thepupil of at least one eye of the driver 3 (and specifically variationsin this diameter), and use it to determine the driver's distractionlevel and/or drowsiness level.

The processing unit 20 may be programmed, when determining the level ofthe driver's inability to drive I_(L), to also take into account one ormore color images 35 delivered by the image-capturing device 1.

The color image 35 may be used in other applications.

The computer 12 may for example transmit the color image 35, or asequence of color images 35, to a telecommunications module 43 of thevehicle 5. This telecommunications module 43 is configured so as totransmit the color image 35, or the received sequence of color images35, to a remote electronic device, for example a multifunction mobile ora computer, for example via a Wi-Fi transmitter. The color image 35 orthe sequence of color images 35 may then be used in the context of ateleconference, a videoconference for example.

The computer 12 could also transmit the color image 35, or the sequenceof color images 35, to a memory of the vehicle 5 for it to be storedtherein.

Various variants may be implemented in the image-capturing device or inthe driver-monitoring system that have been described above.

For example, the second image, which is composed from the pixel signalsgenerated by the second photosensitive pixels of the image sensor, couldbe a monochromic image rather than a color image. The second filterelements of the array of optical filters could moreover all be of thesame type (for example all be green filter elements) rather thancomprising three different types of optical filters (red, green andblue, respectively).

Furthermore, the various functions of the computer could be distributeddifferently between modules. A higher number of modules could be used,or, on the contrary, one and the same module could execute all of theoperations that are executed by the computer and were described above.It should be noted that the term module may designate an electroniccircuit, or a portion of an electronic circuit separate from the othermodules, or a specific group of instructions stored in the memory of thecomputer. Moreover, the preprocessing module could be integrateddirectly into the image sensor.

The computer could also be programmed so that the series of shots takenby the image sensor contains a number of shots different from thatpresented above, for example so that this series contains 3 shotsinstead of two or four. In this case, the series of three shots wouldcontain the first and second shots described above, and an additionalshot, allowing the dynamic range of the brightness values of theinfrared image and/or of the color image finally obtained to beincreased.

The invention claimed is:
 1. An image-capturing device comprising: animage sensor comprising: an array of optical filters receivingelectromagnetic radiation and comprising first optical filters each ableto transmit a first portion of the electromagnetic radiation containedin a given infrared wavelength range, and second optical filters eachable to transmit at least one component of a second portion of theelectromagnetic radiation located in the visible, and a matrix array ofphotosensitive pixels comprising first photosensitive pixels arranged soas to receive the first portion of the electromagnetic radiationtransmitted by the first optical filters, as well as secondphotosensitive pixels arranged so as to receive the componenttransmitted by the second optical filters, each of the first and secondphotosensitive pixels producing via the photoelectric effect an electriccharge or an electric voltage depending on the power of theelectromagnetic radiation that it receives; and a computer programmedto: a1) determine an ambient brightness in the infrared range, a2)determine an ambient brightness in the visible range, b1) command theimage sensor to perform a first image capture, during which each of thefirst photosensitive pixels delivers a pixel signal representative ofthe electric charge or of the electric voltage that it produces, thiselectric charge or voltage having been accumulated by this firstphotosensitive pixel during a first integration time, b2) command theimage sensor to perform a second image capture, during which each of thesecond photosensitive pixels delivers a pixel signal representative ofthe electric charge or of the electric voltage that it produces, thiselectric charge or voltage having been accumulated by this secondphotosensitive pixel during a second integration time, c1) command apreprocessing module to multiply each of the pixel signals produced instep b1) by a first gain value, c2) command the preprocessing module tomultiply each of the pixel signals produced in step b2) by a second gainvalue, d1) compose a first image, from the pixel signals produced by thefirst photosensitive pixels in the first image capture, said pixelsignals having been multiplied by the first gain value, d2) compose asecond image, from the pixel signals delivered by the secondphotosensitive pixels in the second image capture, said pixel signalshaving been multiplied by the second gain value, the first integrationtime or the first gain value being determined by the computer dependingon the ambient brightness in the infrared range, and the secondintegration time or the second gain value being determined by thecomputer depending on the ambient brightness in the visible range. 2.The image-capturing device as claimed in claim 1, wherein the computeris programmed, steps b1), c1) and d1) having been executed beforehand,to determine said ambient brightness in the infrared range, in step a1),depending on brightness values of at least some of the image pixels ofthe first image produced in said previous execution of step d1).
 3. Theimage-capturing device as claimed in claim 2, wherein the computer isprogrammed to, in step a1), determine said ambient brightness in theinfrared range in such a way that it is representative of a firstaverage-brightness level in the first image produced in said previousexecution of step d1).
 4. The image-capturing device as claimed in claim2, wherein the computer is programmed to determine the first integrationtime or the first gain value depending on a first difference between, onthe one hand, a first target average-brightness value, and, on the otherhand, the first average-brightness level in the first image produced inthe previous execution of step d1).
 5. The image-capturing device asclaimed in claim 1, wherein the computer is programmed, steps b2), c2)and d2) having been executed beforehand, to determine said ambientbrightness in the visible range, in step a2), depending on brightnessvalues of at least some of the image pixels of the second image producedin said previous execution of step d2).
 6. The image-capturing device asclaimed in claim 5, wherein the computer is programmed to, in step a2),determine said ambient brightness in the visible range in such a waythat it is representative of a second average-brightness level in thesecond image produced in said previous execution of step d2).
 7. Theimage-capturing device as claimed in claim 5, wherein the computer isprogrammed to determine the second integration time or the second gainvalue depending on a second difference between, on the one hand, asecond target average-brightness value, and, on the other hand, thesecond average-brightness level in the first image produced in theprevious execution of step d2).
 8. The image-capturing device as claimedin claim 1, wherein the computer is further programmed to: b3) commandthe image sensor to perform a third image capture, during which each ofthe first or second photosensitive pixels delivers a pixel signalrepresentative of the electric charge or of the electric voltage that itproduces, this electric charge or voltage having been accumulated by thephotosensitive pixel, during a third integration time, c3) command thepreprocessing module to multiply each of the pixel signals produced instep b3) by a third gain value, the product of the third integrationtime multiplied by the third gain value being different from the productof the first integration time multiplied by the first gain value, orbeing different from the product of the second integration timemultiplied by the second gain value.
 9. The image-capturing device asclaimed in claim 8, wherein the computer is programmed to, in step d1),compose the first image while further taking into account the pixelsignals produced by the first photosensitive pixels in the third imagecapture, these pixel signals having been multiplied by the third gainvalue, the product of the third integration time multiplied by the thirdgain value being different from the product of the first integrationtime multiplied by the first gain value.
 10. The image-capturing deviceas claimed in claim 8, wherein the computer is programmed to, in stepd2), compose the second image while further taking into account thepixel signals produced by the second photosensitive pixels in the thirdimage capture, these pixel signals having been multiplied by the thirdgain value, the product of the third integration time multiplied by thethird gain value being different from the product of the secondintegration time multiplied by the second gain value.
 11. Theimage-capturing device as claimed in claim 9, wherein the computer isfurther programmed to: b4) command the image sensor to perform a fourthimage capture, during which each of the first or second photosensitivepixels delivers a pixel signal representative of the electric charge orof the electric voltage that it produces, this electric charge orvoltage having been accumulated by the photosensitive pixel during afourth integration time, c4) command the preprocessing module tomultiply each of the pixel signals produced in step b4) by a fourth gainvalue, the product of the fourth integration time multiplied by thefourth gain value (g4) being different from the product of the secondintegration time multiplied by the second gain value, the computer beingprogrammed to, in step d2), compose the second image while furthertaking into account the pixel signals produced by the secondphotosensitive pixels in the fourth image capture, these pixel signalshaving been multiplied by the fourth gain value.
 12. The image-capturingdevice as claimed in claim 1, wherein the computer is programmed to, instep d1), compose the first image while further taking into accountpixel signals produced by the first photosensitive pixels in the secondimage capture, the pixel signals produced by the first photosensitivepixels in the second image capture having been multiplied by the secondgain value.
 13. The image-capturing device as claimed in claim 1,wherein the computer is programmed to, in step d2), compose the secondimage while further taking into account pixel signals produced by thesecond photosensitive pixels in the first image capture, the pixelsignals produced by the second photosensitive pixels in the first imagecapture having been multiplied by the first gain value.
 14. A monitoringsystem for monitoring a driver of a vehicle, comprising: animage-capturing device as claimed in claim 1; and a processing unitprogrammed to determine a level of the driver's inability to drive basedat least on said first image.